Internet Dilettantes' Crowd-Based Peer Review: An Exercise in Mediocrity

A suggestion has arisen that posits that a research article peer review process using a crowd-based online protected platform analysis compares favorably with conventional peer review, concluding that it is valid and workable. The suggestion appeared in an observational crowdbased online article filled with guesses, suppositions, and fantasy, justifying a free-for-all peer review. Debate on how alternatives can improve the process, making it more responsive, comprehensive, honest, and ethical and providing an unbiased vetting of scientific reporting would be welcome, but the alternative must meet the standard for scientific rigor. Could be, may be, feels like, and other subjective phrases do not make a change to an article valid. The peer review process gives feedback about misrepresentations, misunderstandings, and the need for clarification. Studies regarding online peer review show that the process has promise, but they have not proved that it is better than the traditional one, nor are they a mandate for change. Self-proclaimed experts with the illusion of possessing knowledge are dangerous for article vetting. Prepublication review should be performed with the utmost attention to detail, the sample size, methodology, statistical errors (such as 1 covariate to 1 response when a multivariate analysis is needed); the use of confidence interval analysis; and validation of the claims made. The reviewer is not the fact finder, but is the fact analyzer and decision maker regarding the text under review. Until there is robust evidence that the current imperfect traditional peer review system will be improved, the search for the holy grail of reviewing processes should continue, but a new approach should not be held to be meaningful if it is not. Peer review is far from perfect, but nothing is perfect. A recent article suggested using “intelligent crowd reviewing,” differentiating it from “crowdsourced” reviewing as a fix for traditional peer review.1 How thin can you slice a bad idea? Let me count the ways. Currently there is the standard peer review process and 8 other methods2 and the 2 proposed Internet alternatives just mentioned: crowdsourced and intelligent crowd reviewing. The standard peer review method raises some concerns, but it has served dissemination of scientific information well, and the proposed alternatives would lead us into the weeds without a repellent against the crowd, intelligent or not.